ci/nightly: manifest aggregator + 90-build retention sweep#2112
Merged
Conversation
Adds the two CI workflows that turn the dated nightly-* releases from PR #2111 into a queryable index served via GitHub Pages. manifest.yml — triggers on successful completion of build.yml (or by manual dispatch). Runs general/scripts/.../enrich_manifest.py to: - enumerate up to 200 GH releases, keep tags matching ^nightly-[0-9]{8}-[0-9a-f]{7}$, sort newest first, keep top 90; - for each, parse the body (sha=/short=/built_at= written by PR #2111) and the assets list, mapping openipc.<soc>-<flash>-<variant>.tgz to platforms.<soc>_<variant>.<flash>; - emit manifest.json (rich, JSON, for hosts/agents/CI) and manifest.flat (whitespace-delimited, for busybox sysupgrade); - commit both to the gh-pages branch. Resulting URLs: - https://openipc.github.io/firmware/manifest.json - https://openipc.github.io/firmware/manifest.flat cleanup.yml — Monday 05:00 UTC + workflow_dispatch. Deletes every dated nightly release beyond the 90 newest via `gh release delete --cleanup-tag`, then re-triggers manifest.yml to drop those entries from the index. A concurrency group `gh-pages-manifest` serializes the manifest and cleanup workflows so concurrent runs cannot race the gh-pages push. md5 is intentionally absent from the v1 schema — each .tgz already ships a .md5sum sidecar that sysupgrade validates after download, so a manifest-level md5 would be redundant and would require downloading every asset on each rebuild. The flat schema is therefore 5 columns: build_id platform flash size url. First-run behaviour: when no dated nightly-* releases exist yet, enrich_manifest writes an explicit empty manifest with a "first cron will populate" comment. Verified locally against the live repo. PR-B of six in the nightly-build redesign. Depends on PR #2111 (merged) for the dated release tag format. The gh-pages branch and Pages site are already configured. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR-B of six in the nightly-build redesign. Adds the two workflows that turn the dated
nightly-*releases from #2111 into a queryable index served via GitHub Pages.manifest.yml— triggers on successfulbuild.ymlcompletion (or manual dispatch). Runs.github/scripts/enrich_manifest.pyto enumerate dated releases, keep the 90 newest, parse the body (sha=/short=/built_at=from ci/nightly: SHA-gate cron, publish dated nightly-* releases, BUILD_ID in os-release #2111) and asset list, then emit:manifest.json— rich JSON for hosts/agents/CI.manifest.flat— whitespace-delimited, parseable by pure busyboxawk(nojqorjsonfilter) for on-devicesysupgradein PR-C.gh-pagesbranch.cleanup.yml— Mondays 05:00 UTC. Deletes dated nightly releases beyond the 90 newest viagh release delete --cleanup-tag, then re-triggersmanifest.yml.concurrency: gh-pages-manifestserializes both workflows so concurrent runs cannot race thegh-pagespush.URLs after merge
(
gh-pagesbranch and Pages site are already configured, pointing atgh-pages//.)Schema notes
md5is intentionally omitted from v1. Each.tgzalready ships a.md5sumsidecar thatsysupgradevalidates after download (general/overlay/usr/sbin/sysupgrade:93-95), so manifest-level md5 would be redundant and would require downloading every asset on each rebuild.manifest.flatcolumns:build_id platform flash size url. Lines starting with#are comments;@channel <name> <build_id>records pin channels.awk '$1==build && $2==plat && $3==flash {print $NF}'is the entire on-device lookup.First-run behaviour
When no dated
nightly-*releases exist yet, the script writes an explicit empty manifest with a "first cron will populate" comment. Already validated locally against the live repo — the empty path works.Test plan
gcc-compatchecks if applicable).manifest.ymlonce; it should write an empty manifest togh-pages(no dated releases exist yet).build.ymlcompletes → publishesnightly-YYYYMMDD-<short>→workflow_runtriggersmanifest.yml→ manifest emerges with one build entry.curl https://openipc.github.io/firmware/manifest.json | jq '.channels.nightly'returns the just-publishedbuild_id.curl https://openipc.github.io/firmware/manifest.flat | awk '\$1==\"<id>\" && \$2==\"hi3520dv200_lite\" && \$3==\"nor\" {print \$NF}'returns the asset URL.cleanup.yml— no-op while ≤90 dated releases exist.🤖 Generated with Claude Code